A note on the Bayesian regret of Thompson Sampling with an arbitrary prior

We consider the stochastic multi-armed bandit problem with a prior distribution on the reward distributions. We show that for any prior distribution, the Thompson Sampling strategy achieves a Bayesian regret bounded from above by 14 √ nK. This result is unimprovable in the sense that there exists a prior distribution such that any algorithm has a Bayesian regret bounded from below by 1 20 √ nK. In this paper we are interested in the Bayesian multi-armed bandit problem which can be described as follows. Let π0 be a known distribution over some set Θ, and let θ be a random variable distributed according to π0. For i ∈ [K], let (Xi,s)s≥1 be identically distributed random variables taking values in [0, 1] and which are independent conditionally on θ. Denote μi(θ) := E(Xi,1|θ). Consider now an agent facing K actions (or arms). At each time step t = 1, . . . n, the agent pulls an arm It ∈ [K]. The agent receives the reward Xi,s when he pulls arm i for the sth time. The arm selection is based only on past observed rewards and potentially on an external source of randomness. More formally, let (Us)s≥1 be an i.i.d. sequence of random variables uniformly distributed on [0, 1], and let Ti(s) = ∑s t=1 1It=i, then It is a random variable measurable with respect to σ(I1, X1,1, . . . , It−1, XIt−1,TIt−1 (t−1), Ut). We measure the performance of the agent through the Bayesian regret defined as